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Abstract: The purpose of this piece is to provide the research 
and rationales behind Practice Profiles. To achieve outcomes 
and develop effective implementation supports, innovations 
need to be “teachable, learnable, doable, and assessable.” 
Practice Profile methodology facilitates the development of 
innovations and their necessary infrastructure. Specific 
training on NIRN’s Practice Profile methodology can be 
found on the Active Implementation Hub. 
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Background 


The field of Implementation Science seeks to determine the 
supporting conditions for the effective implementation of an 
innovation. This might be either a new model of service, an 
evidence-based program, or any of a number of defined 
efforts to create a particular set of outcomes. Implementation 
science can be summarized by the formula: 


Effective x Effective > 4 Enabling =i Positive 
Innovations Implementation Contexts = Outcomes 
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The three factors of the equation refer to what is 
implemented, how it is implemented and where it is 
implemented. In order to achieve significant impact, 
the innovation (the what) needs to be well specified 
and matched to the needs of the population, 
implemented in a deliberate and adaptive manner, as 
well as supported by a hospitable environment and 
learning processes. 


In terms of the “what,” communities are often unable 
to use existing manualized programs to address 
complex and emerging challenges. In these cases, 
communities deliberately choose innovations using 
available knowledge that meet the unique needs of 
the target population, are based on research evidence, 
and are feasible to implement within a specific 
context. In doing so, communities regularly begin 
with conceptually defined _ strategies. When 
innovations lack specification, it is challenging for 
“interventionists” who are left to figure out “what” 
they should be implementing, which results in 
impediments to implementation with good outcomes 
for communities (e.g. Hall & Hord, 2006). 


Practice profiles are a tool for operationalizing a 
conceptually defined strategy through community 
engagement and research methods so that it is clear 
what practitioners will do as they carry out the 
innovation. Once an innovation is described in 
sufficient detail, effective implementation methods 
can be applied to develop the competency of staff to 
do the new way of work, to use data to continuously 
improve the innovation, and to ensure that leadership 
and administrative practices are in service to new 
expectations. Enabling contexts also must be built 
that leverage and build hospitable funding, regulatory 
and policy environments, engage key stakeholders, 
and promote ongoing learning. 


For the innovation to be useful in practice, the following 
criteria are necessary (Metz, Bartley, Blase, Fixsen, 2011; 
Fixsen, Blase, Metz and Van Dyke, 2013): 


e The philosophy, values, and principles that underlie the 
innovation. 
These guide the practitioner’s decisions and ensure 
consistency, integrity, and sustainable effort across all 
practitioners. 

e Clear description of the essential functions. 
These define the role of practitioners and inform activities 
within each phase of work. Essential functions provide a 
clear description of the features that must be present to say 
that the innovation is being used and to achieve outcomes 
(“essential functions” are sometimes called core 
components, active ingredients, or practice elements). 

e Operational definitions of the essential functions. 
These describe the core activities associated with each 
essential function and allow the innovation to be “teachable, 
learnable, doable, and assessable” across a range of 
contexts. Operational definitions promote functional 
consistency across practitioners at the service delivery level. 

e Practical assessments of performance. 
This assesses whether the innovation is implemented as 
intended. Fidelity assessments are used to improve 
practitioner competency and implementation supports such 
as training and coaching. 


Practice Profile Methodology 


The development of practice profiles requires a specific 
methodology. The methodology ensures the inclusion of 
research or best practices, the alignment of competencies with 
the innovation’s theory of change, and the recognition of 
“what works” through the experience of communities, 
practitioners and key stakeholders. Implementation teams 
conduct the following interrelated (sometimes overlapping) 
steps in an iterative process to identify the principles, 
essential functions, and activities of practitioners: 1) semi- 
structured interviews; 2) systematic scoping review; 3) 
document review; 4) vetting and consensus process; 5) 
usability testing. These steps are described in greater in the 
following section. 
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Semi-Structured Interviews 


Individual interviews are conducted with a sample of 
practitioners and community members engaged in 
services. The goal is to identify the innovation’s 
principles that guide successful work with children, 
youth, adults, and families, as well as the specific 
activities practitioners are engaged with to bring 
these principles to life. Practitioners and community 
members are asked to provide examples from the 
field to illustrate the use of guiding principles and 
core activities related to the innovation. Practitioners 
also are asked to consider successes and challenges 
in implementing the innovation. Community 
members are asked to consider the benefits and 
challenges of innovations in supporting their desired 
outcomes. Other key stakeholders are _ also 
interviewed as needed. Findings from the interviews 
are coded for themes to inform the development of a 
draft description of the practice profile. 


Document Review 


Existing documentation of the innovation is also 
reviewed. This can include program theory, logic 
model, program description, communication plans, 
and other tools and resources (e.g., CQI and 
monitoring tools, quarterly reports, and site visit 
reports). The document review has two purposes: 1) 
to select documents based on whether they contain 
information needed to develop the interview protocol 
and scoping review questions and search terms; and 
2) to describe in greater detail information related to 
principles and essential functions to inform the 
practice profile development. 


Systematic Scoping Review 


Scoping reviews (Levac, Colquhoun, & O’Brien, 
2010) allow for a rapid and systematic review of 
published work in a broad thematic area. The goal of 
the scoping review is to access and review published 
research that focuses on identifying competencies 
related to the innovation. The scoping review 
includes six stages (Arksey & O’Malley, 2005): 1) 
identifying the research question for the scoping 
review to address; 2) identifying relevant studies and 
reports; 3) selecting studies and reports using post 


hoc inclusion and exclusion criteria based on 
increasing familiarity with the literature; 4) 
extracting data to capture process oriented 


information; 5) summarizing and reporting results; 
and 6) consulting with community members and key 
stakeholders to request additional insights beyond the 
published frameworks. Studies and articles are 
identified through literature searches and a 
snowballing technique involving key sources such as 


developers or implementers of the innovation. Themes are 
summarized and integrated with findings from the qualitative 
interviews to inform the practice profile development. 


Vetting and Consensus Building 


Community members, practitioners, leadership, and other key 
stakeholders (researchers, program consultants) vet the initial 
draft of the practice profile. Facilitation and guiding 
questions are provided for the vetting and consensus building 
process. This process happens in two phases and typically 
takes place over the course of several meetings. The first 
phase of vetting and consensus building provides an 
opportunity for stakeholder reflection. Each stakeholder is 
asked to respond to the following questions: 1) What are your 
thoughts after reading through the profile? 2) What do you 
see as the strengths in the profile? 3) What do you notice as 
areas to be improved or more fully realized? 4) Did you 
notice anything missing? In the second phase of vetting and 
consensus building, stakeholders are asked to provide 
feedback on each essential function. Questions include: 1) Is 
this an essential function for practitioners? 2) If no, should it 
be included within another essential function or removed? 3) 
If yes, are the core activities measurable and observable? 4) 
What changes or additions are recommended? 5) What 
additional literature should be reviewed? 6) Are more 
perspectives needed? Once consensus is developed, the 
practice profile is tested with real world cases. 


Testing and Evolving the Profile 


Usability testing uses rapid cycle (Plan, Do, Study, Act) 
detection of strengths and gaps related to the evolving 
innovation with a small samples of cases. By “testing” the 
innovation as it is expected to be implemented with only a 
few examples (e.g., three to five practitioners initiating new 
services) across agencies/counties/regions, improvements can 
be made quickly from one cycle to the next. Data are 
synthesized across practitioners to provide feedback on the 
overall usability of the practice profile. When used 
purposefully, 4 or 5 usability testing cycles with 4 or 5 
practitioners involved in each cycle can produce information 
needed to refine the practice and get started on purposeful 
continuous quality improvement strategies as the innovation 
is scaled within the practice setting. When consistent 
challenges occur, implementation supports or the profile itself 
may be adjusted or contextualized to represent “real world” 
implementation. Reflection, problem solving, and small 
cyclical tests of change are the hallmark of this phase of 
practice profile development. 
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Practice profile methodology is aligned with 
continuous quality improvement — optimizing the 
use of evidence in a range of contexts and enabling 
ongoing learning among practitioners, researchers, 
policy makers and funders, and community members 
to improve the sustainability of evidence in practice 
settings. Operational learning (Chambers, Glasgow, 
& Stange, 2013) is a core value of the practice profile 
methodology. As described by Damschroder and 


colleagues th 


e 


successful implementation of 


innovations includes “dedicated time for reflecting or 
debriefing before, during, and after implementation 
as one way to promote shared learning and 
improvements along the way” (Damschroder et al., 
2009, pg. 11). 
has implications for researchers, 
practitioners and policy makers. For researchers, 
development and refinement of interventions are not 
completed during clinical trials, but rather optimized 
through ongoing use in practice settings. Policy 
makers will need to assess opportunities to 
incentivize ongoing, data-driven improvement 
strategies. Practitioners will need to employ 
feedback loops 
researchers and policy makers to continually assess 
and improve practice (Chambers, et. al., 2013). 


improvement 


Practice _ profile 
commitment to adapting models and practices to 
achieve more desirable outcomes (see Kainz and 
Metz, 2016). While there is increasing emphasis on 
the importance 
optimizing evidence-based practices and programs in 
local contexts, there is less information on how to 
assess “fit” and determine optimal adaptations to 
evidence-based models without compromising 
outcomes. Conceptual models for guiding adaptation 
(Aarons, 2012) shed some light on how to tailor 
systems, organizations, and programs to meet the 
needs of local communities and target populations. 
The practice profile methodology provides a concrete 
strategy for factoring in the dynamic interplay 
between characteristics of the service system, service 
delivery organization, and communities. This is done 
by including key stakeholders in the co-creation of 
the innovation, 
implementation and improvement. 
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Benefits of Practice Profiles 
Practice profiles have many, including: 


e Support co-creation of the innovation 
Practice profiles include key stakeholders (community 
members, practitioners, researchers and policy makers) in 
the operationalization, execution, and improvement of the 
innovation and provide consensus-building opportunities. 

e Provide a fully operationalized practice model for 
consistent implementation of the innovation 
Practice profiles provide greater specificity of the 
innovation and improve the likelihood that practitioners can 
engage in the essential functions. 

e Facilitate the development of effective training protocols, 
coaching strategies, and fidelity assessments 
A well-operationalized innovation allows for the 
development of competency-based recruiting, hiring, 
training, coaching, and fidelity assessments that are “in 
service to” the essential functions outlined in the practice 
profile. Without this level of operationalization, 
organizations will be unable to develop the right supports 
and infrastructure for ensuring the innovation is used 
effectively and improved over time. 

e Refine the organizational and systems supports the 
organization or agency will need to install to facilitate 
consistent and effective practice across practitioners 
A well-operationalized innovation will allow organizations 
to develop decision-support data systems, administrative 
practices, and systems partnerships aligned with the 
expectations for the new way of work. 

e Promote the use of continuous improvement as an essential 
functions of the practice model 
Organizations can only improve innovations that are well- 
defined. Without knowing what “it” is, “it” cannot be 
tested (e.g., in interactions with children and families) and 
improved over time. Usability testing allows for data-driven 
enhancements and revisions to the profile, leading to 
improved practice by practitioners. 

e Ensure that outputs and outcomes of the innovation can be 
accurately interpreted 
Outcomes are challenging to interpret when there is a lack 
of clarity in “what” was implemented. If expected 
outcomes are not achieved, fidelity assessment data related 
to the practice profile components can provide an 
explanation and facilitate action planning. 

e Demonstrate a commitment to adapting models and 
practices to achieve more desirable outcomes 
Practice profiles emphasize the importance of translating, 
adapting, and optimizing evidence-based practices and 
programs in local contexts. 
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